Data storage system and log data output method upon abnormality of storage control apparatus

ABSTRACT

A storage system have a plurality of control modules which control a plurality of disk storage devices, in which system information reading/writing is possible even when problems arise in paths with a plurality of disk devices, and moreover log data can be output even upon occurrence of an abnormality in a control module. A plurality of control modules has built-in a pair of system disk device units which store log data. Upon occurrence of an abnormality in one control module, a system disk unit of one control module is removed and mounted in another control module, and the log data of the mounted system disk unit is output by the another control module.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromthe prior Japanese Patent Application No. 2005-085285, filed on Mar. 24,2005, the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to a data storage system used as an externalstorage apparatus for a computer and to a log data output method uponoccurrence of an abnormality for a storage control apparatus, and inparticular relates to a data storage system having, among numerous diskdevices, a disk device used by a user and a system disk device used bythe apparatus, and to an output method upon occurrence of an abnormalityfor a storage control apparatus.

2. Description of the Related Art

As data has assumed various electronic forms in recent years and hascome to be handled by computers, independently of host computersexecuting data processing, data storage apparatuses (external storageapparatuses) capable of storing large amounts of data efficiently andwith high reliability have become increasingly important.

As such data storage apparatus, disk array apparatuses comprisinglarge-capacity disk devices (for example, magnetic disk and optical discdevices) and disk controllers used to control such large-capacity diskdevices have come into use. Such disk array apparatuses have memoryserving as a disk cache. By this means, when read requests and writerequests are received from a host computer, the time required to accessdata can be shortened, and enhanced performance can be achieved.

In general, a disk array apparatus has a plurality of principal units,that is, channel adapters which are portions for connection to hostcomputers, disk adapters which are portions for connection to diskdrives, memory having a cache area, a control unit which serves tocontrol the cache memory, and large-capacity disk drives.

FIG. 20 explains the technology of the prior art. The disk arrayapparatus 102 shown in FIG. 20 comprises two control managers (memory,including cache memory, and a control portion) 10; each control manager10 is connected to channel adapters 11 and disk adapters 13.

The two control managers 10, 10 are directly connected by a bus 10 c soas to enable communication. The channel adapters 11 are connected tohost computers (not shown) by for example fiber channel or Ethernet (aregistered trademark). The disk adapters 13 are connected to each of thedisk drives in disk enclosures 12 by, for example, fiber channel cable.

A disk enclosure 12 has two ports (for example, fiber channel ports);these two ports are connected to different disk adapters 13. By thismeans redundancy is imparted, and fault tolerance is improved. (See forexample Japanese Patent Laid-open No. 2001-256003)

In such a large-capacity data storage system, a large amount ofinformation (called system information) is necessary for control bycontrollers (control units, channel adapters, disk adapters andsimilar). For example, system information includes firmware necessary tooperate controllers, backup data for the apparatus configuration, andlog data for various tasks and threads.

The firmware comprises control programs for controllers; in particular,in a disk array (RAID configuration), numerous control programs arenecessary. Backup data for the apparatus configuration is data used toconvert from host-side logical addresses to physical disk addresses; alarge amount of data is necessary, according to the number of diskdevices and number of hosts. Log data is state data for each task andthread, used for fault recovery and fault prevention, and alsoconstitutes a large volume of data.

Such system data is generally stored in a nonvolatile large-capacitystorage device; in the prior art, as shown in FIG. 20, one or more ofthe disk drives 120 in the disk enclosure 12 connected by cables to thedisk adapters 13 was used for storage of such data. A disk drive whichstores this system data is called a system disk.

That is, a portion of the numerous disk drives connected to controllersare used as system disks, and the other disk drives are used as userdisks. As a consequence of this conventional technology, as indicated inFIG. 20, any of the controllers 10 can access system disks 120.

However, in addition to redundancy, in recent years storage systems havebeen required to continue operation even upon occurrence of a fault inany portion of the system. In the technology of the prior art, if aproblem arises in the path between a controller and a disk enclosure,such as for example between a disk adapter and a disk enclosure, readingand writing of the system disk 120 can no longer be executed.

Consequently even if the controller and other paths are normal, thecontroller cannot read firmware or apparatus configuration backup datafrom the system disk, and operations using other routes becomedifficult. Further, the controller cannot read or write log data to andfrom the system disk, impeding analysis upon occurrence of a fault anddiagnostics for fault prevention.

Moreover, upon occurrence of a power outage it is necessary to switch tobattery operation and to back up the data in cache memory to the systemdisk. In the technology of the prior art, in such cases power must alsobe supplied to the disk enclosure, so that a very large battery capacityis required. Furthermore, a comparatively long time is necessary towrite backup data to a system disk via a disk adapter and cable, andwhen the cache memory capacity is large, a huge battery capacity isrequired.

SUMMARY OF THE INVENTION

Hence an object of this invention is to provide a data storage systemand a log data output method upon occurrence of an abnormality for adata storage control apparatus, which can execute reading/writing of asystem disk even when problems occur in a path between a controller anda group of disk drives, and enabling output of system disk log data evenupon occurrence of an abnormality in the controller.

A further object of this invention is to provide a data storage systemand a log data output method upon occurrence of an abnormality for adata storage control apparatus, enabling smaller battery capacity forbackups during power outages, with an inexpensive configuration, andwhich can output system disk log data even upon occurrence of anabnormality in the controller.

Still another object of this invention is to provide a data storagesystem and a log data output method upon occurrence of an abnormalityfor a data storage control apparatus, enabling backups of cache memorydata with a small battery capacity during power outages, and which canoutput system disk log data even upon occurrence of an abnormality inthe controller.

In order to attain these objects, a data storage system of thisinvention has a plurality of disk storage devices which store data and aplurality of control modules, connected to the plurality of disk storagedevices, which control access to the disk storage devices, according toaccess instructions from a higher-level system. And each of the controlmodules has memory having a cache area which stores a portion of thedata stored in the disk storage devices, a control unit which controlsaccess, a first interface portion which controls the interface with thehigher-level system, a second interface portion which controls theinterface with the plurality of disk storage devices, and a pair ofsystem disk units, connected to the control unit, which store, at least,log data of the control unit. Further one control module, upon theoccurrence of an abnormality in one of the other control modules,detects that one system disk unit of the other control module isinserted into a system disk slot of the one control module, incorporatesthe one system disk unit of the other control module, and outputs thelog data of the one system disk unit of the other control module.

Further, a log data output method upon occurrence of an abnormality fora storage control apparatus of this invention is a log data outputmethod upon occurrence of an abnormality for a storage controlapparatus, connected to a plurality of disk storage devices which storedata, and having a plurality of control modules which control access ofthe disk storage devices according to access instructions from ahigher-level system, where each of the control modules has memory havinga cache area which stores a portion of the data stored in the diskstorage devices, a control unit which controls access, a first interfaceportion which controls the interface with the higher-level system, asecond interface portion which controls the interface with the pluralityof disk storage devices, and a pair of system disk units, connected tothe control unit, which store, at least, log data of the control unit.The output method has a step, upon an abnormality in another controlmodule, of detecting that one system disk unit removed from the othercontrol module is inserted into a system disk slot of one controlmodule; a step of incorporating the one system disk unit of the othercontrol module into the one control module; and a step of outputting thelog data of the incorporated one system disk unit of the other controlmodule using the one control module.

In this invention, it is preferable that the one control module, afterincorporating the one system disk unit of the other control module, copythe log data of the system disk unit of the one control module to theincorporated system disk unit, without destroying the log data of theother control module in the system disk unit of the other controlmodule.

In this invention, it is preferable that the one control module read anidentifier of the inserted system disk unit, and judge whether the onesystem disk unit of the other control module has been incorporated.

In this invention, it is preferable that when the one control unitjudges, from the identifier, that the inserted system disk unit is notthe one system disk unit of the other control module, the one controlunit copy the log data of the system disk unit of the one control moduleto the incorporated system disk unit.

In this invention, it is preferable that the one control module read thelog data area of the incorporated system disk unit of the other controlmodule, and copy, to a system disk unit area other than the log dataarea of the other control module, the log data of the system disk unitof the one control module.

In this invention, it is preferable that the one control module separatethe other system disk unit of the one control module, in response to aninstruction from an external apparatus, and release the system disk slotto enable insertion of the system disk unit of the other control module.

In this invention, it is preferable that the one control module outputthe log data of the other control module from the incorporated systemdisk unit, in response to a log data acquisition instruction from anexternal apparatus.

In this invention, it is preferable that the one control module outputthe log data of the other control module to the external apparatus.

A system disk is built into the control module, so that even if aproblem arises in a path between the control module and disk storagedevices, if the control module and other paths are normal, the controlmodule can read firmware and apparatus configuration backup data fromthe system disk, and operations using other paths are possible;moreover, log data can be read and written, so that analysis uponoccurrence of a fault and diagnostics for fault prevention are possible.

Further, when in the event of a power outage the power is switched tobatteries and the data in cache memory is backed up to a system disk,there is no need to supply power to a connected disk storage device, sothat the battery capacity can be made small.

And, even if an abnormality occurs in the other control module, thesystem disk drive of the other control module can be inserted into onecontrol module and reading performed by the one control module, so thateven if a system disk drive is built into a control module, log data ofthe system disk of the abnormal control module can be output.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows the configuration of the data storage system of oneembodiment of the invention;

FIG. 2 shows the configuration of a control module in FIG. 1;

FIG. 3 shows the configuration of a back-end router and disk enclosurein FIG. 1 and FIG. 2;

FIG. 4 shows the configuration of a disk enclosure in FIG. 1 and FIG. 3;

FIG. 5 explains read processing in the configuration of FIG. 1 and FIG.2;

FIG. 6 explains write processing in the configuration of FIG. 1 and FIG.2;

FIG. 7 shows the mounted configuration of a control module in oneembodiment of the invention;

FIG. 8 shows an example of the mounted configuration of a data storagesystem in one embodiment of the invention;

FIG. 9 explains the operation of removing a disk of an abnormalcontroller in the log data output method of one embodiment of theinvention;

FIG. 10 explains the operation of disk separation in the log data outputmethod of one embodiment of the invention;

FIG. 11 explains normal controller disk removal operation in the logdata output method of one embodiment of the invention;

FIG. 12 explains disk insertion operation in the log data output methodof one embodiment of the invention;

FIG. 13 explains log data redundancy operation in the log data outputmethod of one embodiment of the invention;

FIG. 14 explains log data output operation in the log data output methodof one embodiment of the invention;

FIG. 15 shows the flow of log data output processing in one embodimentof the invention;

FIG. 16 explains the configuration information definition table in oneembodiment of the invention;

FIG. 17 shows the flow of information extraction processing in FIG. 15;

FIG. 18 explains disk exchange operation in the maintenance and exchangeprocessing of FIG. 15;

FIG. 19 explains log data redundancy operation in the maintenance andexchange processing of FIG. 15; and

FIG. 20 shows the configuration of a storage system of the prior art.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Below, embodiments of the invention are explained, in the order of adata storage system, read/write processing, mounted configuration, logdata output method upon abnormality, log data output processing uponabnormality, and other embodiments.

Data Storage System

FIG. 1 shows the configuration of the data storage system of oneembodiment of the invention, FIG. 2 shows the configuration of a controlmodule in FIG. 1, FIG. 3 shows the configuration of a back-end routerand disk enclosure in FIG. 1, and FIG. 4 shows the configuration of adisk enclosure in FIG. 1 and FIG. 3.

FIG. 1 shows a mid-scale disk array apparatus having four controlmodules, as an example of a data storage system. As shown in FIG. 1, thedisk array apparatus 1 has a plurality of disk enclosures 2-0 to 2-15holding data; a plurality (here, four) of control modules 4-0 to 4-3,positioned between a host computer (data processing system), not shown,and the plurality of disk enclosures 2-0 to 2-15; a plurality (here,four) of back-end routers (first switch units; hereafter “BRTs”) 5-0 to5-3, provided between the plurality of control modules 4-0 to 4-3 andthe plurality of disk enclosures 2-0 to 2-15; and a plurality (here,two) of front-end routers (second switch units; hereafter “FRTs”) 6-0,6-1.

Each of the control modules 4-0 to 4-3 has a controller 40, channeladapter (first interface portion; hereafter “CA”) 41, disk adapters(second interface portions; hereafter “DAs”) 42 a, 42 b, and DMA (DirectMemory Access) engine (communication portion; hereafter “DMA”) 43.

In FIG. 1, to simplify the drawing, the controller symbol “40”, diskadapter symbols “42 a” and “42 b”, and DMA symbol “43” are assigned onlyto the control module 4-0, and symbols are omitted for the constituentcomponents of the other control modules 4-1 to 4-3.

The control modules 4-0 to 4-3 are explained using FIG. 2. Thecontrollers 40 perform read/write processing based on processingrequests (read requests or write requests) from a host computer, and hasa memory 40 b, a control unit 40 a, and a system disk drive unit 40 c.

The memory 40 b has a cache area, which serves as a so-called cache fora plurality of disks, holding a portion of the data held in theplurality of disks of the disk enclosures 2-0 to 2-15; a configurationdefinition storage area 470; and other work areas.

The control unit 40 a controls the memory 40 b, channel adapters 41,device adapters 42, and DMA 43, and has one or a plurality (here, two)of CPUs 400, 410, and a memory controller 420. The memory controller 420controls memory reading and writing, and also performs path switching.

The memory controller 420 is connected via a memory bus 434 to thememory 40 b, via CPU bus 430, 432 to the CPUs 400, 410, and viafour-lane high-speed serial buses (for example, PCI-Express) 440, 442 tothe disk adapters 42 a, 42 b.

Similarly, the memory controller 420 is connected via four-lanehigh-speed serial buses (for example, PCI-Express) 443, 444, 445, 446 tothe channel adapters 41 (here, four channel adapters 41 a, 41 b, 41 c,41 d), and via four-lane high-speed serial buses (for example,PCI-Express) 447, 448 to the DMA units 43 (here, two DMA units 43-a,43-b).

The PCI-Express or other high-speed serial buses perform packetcommunication, and by providing a plurality of lanes in the serialbuses, the number of signal lines can be reduced with minimal delays andfast response, in so-called low-latency communication.

Further, the memory controller 420 is connected via the serial bus 436to the system disk drive unit 40 c. The system disk drive unit 40 c hasa bridge circuit 450, a fiber channel circuit 452, and a pair of systemdisk drives 453, 454.

The bridge circuit 450 connects the memory controller 420 to the fiberchannel circuit 452 and to a service processor 44 provided on theoutside of the control module 4-0. The service processor 44 comprises,for example, a personal computer, and is used for system stateconfirmation, diagnostics and maintenance.

The fiber channel circuit 452 is connected to at least two system diskdrives 453, 454 (here, two hard disk drives). Hence the CPUs 400, 410and similar can directly access the system disk drives 453, 454 via thememory controller 420. Further, the service processor 44 also can accessthe system disk drives 453, 454, via the bridge circuit 450.

The two system disk drives 453, 454 mirror the log data and similar.That is, a copy of the data on one of the system disk drive 453 isstored in the other system disk drive 454. In other words, the systemdisk drives 453, 454 are built into the control module 4-0, and the CPUs400, 410 can access the system disk drives 453, 454 without theintervention of the DAs 42 a, 42 b or BRT 5-0.

The channel adapters 41 a to 41 d are interfaces with host computers andthe channel adapters 41 a to 41 d are each connected to a different hostcomputer. It is preferable that the channel adapters 41 a to 41 d areeach connected to the interface portions of the corresponding hostcomputers via a bus, such as for example a fiber channel or Ethernet (aregistered trademark) bus; in this case, an optical fiber or coaxialcable is used as the bus.

Further, the channel adapters 41 a to 41 d are each configured as aportion of the control modules 4-0 to 4-3. These channel adapters 41 ato 41 d support a plurality of protocols as the interfaces with thecorresponding host computers and the control modules 4-0 to 4-3.

Because protocols to be mounted are not the same, depending on the hostcomputers supported, the controllers 40 which are the principal units ofthe control modules 4-0 to 4-3 are mounted on separated print boards, sothat the channel adapters 41 a to 41 d can be replaced easily asnecessary.

For example, protocols with host computers to be supported by thechannel adapters 41 a to 41 d include, as described above, fiber channeland iSCSI (Internet Small Computer System Interface) supporting Ethernet(a registered trademark).

Further, as explained above, each of the channel adapters 41 a to 41 dis directly connected to the controller 40 by the bus 443 to 446, suchas a PCI-Express bus, designed for connection of LSI (Large ScaleIntegrated) devices and print boards. By this means, the high throughputrequired between the channel adapters 41 a to 41 d and the controllers40 can be achieved.

The disk adapters 42 a, 42 b are interfaces with each of the disk drivesin the disk enclosures 2-0 to 2-15, and are connected to the BRTs 5-0 to5-3 connected to the disk enclosures 2-0 to 2-15; here, the diskadapters have four FC (Fiber Channel) ports.

As explained above, each of the disk adapters 42 a, 42 b is connecteddirectly to the controller 40 by a bus, such as a PCI-Express bus,designed for connection to LSI (Large Scale Integrated) devices andprint boards. By this means, the high throughput required between thedisk adapters 42 a, 42 b and the controllers 40 can be achieved.

As shown in FIG. 1 and FIG. 3, the BRTs 5-0 to 5-3 are multi-portswitches which selectively switch the disk adapters 42 a, 42 b of thecontrol modules 4-0 to 4-3 and each of the disk enclosures 2-0 to 2-15and make connections enabling communication.

As shown in FIG. 3, each of the disk enclosures 2-0 to 2-7 is connectedto a plurality (here, two) of BRTs 5-0, 5-1. As shown in FIG. 4, aplurality (here, 15) of disk drives 200, each having two ports, areinstalled in each of the disk enclosures 2-0 to 2-7. The disk enclosure2-0 is configured with the necessary number of unit disk enclosures 20-0to 23-0, each having four connection ports 210, 212, 214, 216, connectedin series, to obtain increased capacity. Here, up to a maximum four unitdisk enclosures 20-0 to 23-0 can be connected.

Within each of the unit disk enclosures 20-0 to 23-0, each port of eachdisk drive 200 is connected to two ports 210, 212 by means of a pair ofFC cables from the two ports 210, 212. As explained in FIG. 3, these twoports 210, 212 are connected to different BRTs 5-0, 5-1.

As shown in FIG. 1, each of the disk adapters 42 a, 42 b of the controlmodules 4-0 to 4-3 are connected to all the disk enclosures 2-0 to 2-15.That is, the disk adapters 42 a of each of the control modules 4-0 to4-3 are connected to BRT 5-0 (see FIG. 3) connected to the diskenclosures 2-0 to 2-7, BRT 5-0 connected to the disk enclosures 2-0 to2-7, BRT 5-2 connected to the disk enclosures 2-8 to 2-15, and BRT 5-2connected to the disk enclosures 2-8 to 2-15.

Similarly, the disk adapters 42 b of each of the control modules 4-0 to4-3 are connected to BRT 5-1 (see FIG. 3) connected to the diskenclosures 2-0 to 2-7, BRT 5-1 connected to the disk enclosures 2-0 to2-7, BRT 5-3 connected to the disk enclosures 2-8 to 2-15, and BRT 5-3connected to the disk enclosures 2-8 to 2-15.

In this way, each of the disk enclosures 2-0 to 2-15 is connected to aplurality (here, two) of BRTs, and different disk adapters 42 a, 42 b inthe same control modules 4-0 to 4-3 are connected to the two BRTsconnected to the same disk enclosures 2-0 to 2-15.

By means of such a configuration, each control module 4-0 to 4-3 canaccess all of the disk enclosures (disk drives) 2-0 to 2-15 via eitherof the disk adapters 42 a, 42 b, and via any path.

As shown in FIG. 2, each disk adapter 42 a, 42 b is connected to thecorresponding BRT 5-0 to 5-3 by a bus, such as for example a fiberchannel or Ethernet (a registered trademark) bus. In this case, asexplained below, the bus is provided as electrical wiring on the printboard of the back panel.

As explained above, one-to-one mesh connections are provided between thedisk adapters 42 a, 42 b of each of the control modules 4-0 to 4-3 andthe BRTs 5-0 to 5-3 to connect all the disk enclosures, so that as thenumber of control modules 4-0 to 4-3 (that is, the number of diskadapters 42 a, 42 b) increases, the number of connections increases andconnections become complex, so that physical mounting becomes difficult.However, by adopting fiber channels, requiring few signals to constructan interface, as the connections between the disk adapters 42 a, 42 band the BRTs 5-0 to 5-3, mounting on the print board becomes possible.

When each of the disk adapters 42 a, 42 b and corresponding BRTs 5-0 to5-3 are connected by a fiber channel, the BRTs 5-0 to 5-3 are fiberchannel switches. Further, the BRTs 5-0 to 5-3 and the correspondingdisk enclosures 2-0 tot 2-15 are for example connected by fiberchannels; in this case, because the modules are different, connection isby optical cables 500, 510.

As shown in FIG. 1, the DMA engines 43 communicate with each of thecontrol modules 4-0 to 4-3, and handle communication and data transferprocessing with the other control modules. Each of the DMA engines 43 ofthe control modules 4-0 to 4-3 is configured as a portion of the controlmodules 4-0 to 4-3, and is mounted on the board of the controller 40which is a principal unit of the control modules 4-0 to 4-3. Each DMAengine is directly coupled to the controllers 40 by means of thehigh-speed serial bus described above, and also communicates with theDMA engines 43 of the other control modules 4-0 to 4-3 via the FRTs 6-0,6-1.

The FRTs 6-0, 6-1 are connected to the DMA engines 43 of a plurality (inparticular three or more; here, four) of control modules 4-0 to 4-3,selectively switch among these control modules 4-0 to 4-3, and makeconnections enabling communication.

By means of this configuration, each of the DMA engines 43 of thecontrol modules 4-0 to 4-3 executes communication and data transferprocessing (for example, mirroring processing) via the FRTs 6-0, 6-1between the controller 40 to which it is connected and the controllers40 of other control modules 4-0 to 4-3, according to access requests andsimilar from a host computer.

Further, as shown in FIG. 2, the DMA engines 43 of each control module4-0 to 4-3 comprise a plurality (here, two) of DMA engines 43-a, 43-b;each of these DMA engines 43-a, 43-b uses two FRTs 6-0, 6-1.

As indicated in FIG. 2, the DMA engines 43-a, 43-b are connected to thecontroller 40 by, for example, a PCI-Express bus. That is, incommunication and data transfer (DMA) processing between the controlmodules 4-0 to 4-3 (that is, between the controllers 40 of the controlmodules 4-0 to 4-3), large amounts of data are transferred, and it isdesirable that the time required for transfer be short, so that a highthroughput as well as low latency (fast response time) are demanded.Hence as shown in FIG. 1 and FIG. 2, the DMA engines 43 and FRTs 6-0,6-1 of the control modules 4-0 to 4-3 are connected by a bus whichutilizes high-speed serial transfer bus (PCI-Express or Rapid-IO)designed so as to satisfy the demands for both high throughput and lowlatency.

The PCI-Express and Rapid-IO buses employ high-speed serial transfer at2.5 Gbps; a small-amplitude differential interface called LVDS (LowVoltage Differential Signaling) is adopted as the bus interface.

Read/Write Processing

Next, read processing in the data storage system of FIG. 1 through FIG.4 is explained. FIG. 5 explains read operation in the configuration ofFIG. 1 and FIG. 2.

First, when a control unit (control manager) 40 receives a read requestvia a channel adapter 41 a to 41 d from one of the corresponding hostcomputers, if the relevant data of the read request is held in the cachememory 40 b, the relevant data held in the cache memory 40 b is sent tothe host computer via the channel adapter 41 a to 41 d.

If on the other hand the relevant data is not held in the cache memory40 b, the control manager (control portion) 40 a first reads therelevant data from the disk drive 200 holding the relevant data into thecache area of memory 40 b, and then transmits the relevant data to thehost computer issuing the read request.

Processing to read the disk drive is explained in FIG. 5.

(1) The control unit 40 a (CPU) of the control manager 40 creates a FCheader and descriptor in the descriptor area of cache memory 40 b. Adescriptor is a command requesting data transfer by a data transfercircuit, and contains the address in the cache memory of the FC header,the address in the cache memory of the data to be transferred, thenumber of data bytes, and the logical address of the disk for datatransfer.

(2) The data transfer circuit of the disk adapter 42 is started.

(3) The started data transfer circuit of the disk adapter 42 reads thedescriptor from the cache memory 40 b.

(4) The started data transfer circuit of the disk adapter 42 reads theFC header from the cache memory 40 b.

(5) The started data transfer circuit of the disk adapter 42 decodes thedescriptor and obtains the request disk, leading address, and number ofbytes, and transfers the FC header to the relevant disk drive 200 usingthe fiber channel 500 (510). The disk drive 200 reads the requesteddata, and transmits the data over the fiber channel 500 (510) to thedata transfer circuit of the disk adapter 42.

(6) Upon having read and transmitted the requested data, the disk drive200 transmits a completion notification over the fiber channel 500 (510)to the data transfer circuit of the disk adapter 42.

(7) Upon receiving the completion notification, the data transfercircuit of the disk adapter 42 reads the read data from the memory ofthe disk adapter 42 and stores the data in the cache area of memory 40b.

(8) When read transfer is completed, the started data transfer circuitof the disk adapter 42 uses an interrupt to send completion notificationto the control manager 40.

(9) The control unit 40 a of the control manager 40 obtains theinterrupt source of the disk adapter 42 and confirms the read transfer.

(10) The control unit 40 a of the control manager 40 checks the endpointer of the disk adapter 42 and confirms the completion of readtransfer.

Thus in order to obtain sufficient performance, high throughput must bemaintained over all connections, but many signals (here, seven) areexchanged between the control unit 40 a and disk adapter 42, and alow-latency bus is especially important. In this embodiment, both thePCI-Express (four-lane) bus and the Fiber Channel (4G) bus are adoptedas connections having high throughput; but whereas PCI-Express is alow-latency connection, Fiber Channel is a comparatively high latency(time is required for data transfer) connection.

In this embodiment, fiber channel can be adopted in the BRTs 5-0 to 5-3for the configuration of FIG. 1. In order to achieve low latency,although the number of bus signals cannot be decreased beyond a certainnumber, in this embodiment fiber channel with a small number of signallines can be used for the connection between disk adapters 42 and BRTs5-0; the number of signals on the back panel is reduced, providingadvantages for mounting.

Next, write operation is explained. When a write request is receivedfrom one of the host computers via the corresponding channel adapter 41a to 41 d, the channel adapter 41 a to 41 d which has received the writerequest command and write data queries the control manager 40 for theaddress in the cache area of memory 40 b to which to write the writedata.

When the channel adapter 41 a to 41 d receives the response from thecontrol manager 40, the write data is written to the cache area ofmemory 40 b of the control manager 40, and in addition the write data iswritten to the cache area in the memory 40 b in at least one controlmanager 40 different from the control manager 40 in question (that is,the control manager 40 of a different control module 4-0 to 4-3). Forthis purpose the DMA engine 43 is started, and the write data is alsowritten to the cache area of memory 40 b in the control manager 40 ofanother control module 4-0 to 4-3, via an FRT 6-0, 6-1.

Here, by means of redundant writing (mirroring) of the data, even in theevent of an unforeseen hardware failure of a control module 4-0 to 4-3or control manager 40, data loss can be prevented. Finally, when writingof cache data to the cache areas of the plurality of memory units 40 bends normally, the channel adapter 41 a to 41 d sends notification ofcompletion to the host computer, and processing ends.

The write data must then be written back (write-back) to the relevantdisk drive. The control unit 40 a writes back the write data in thecache area of memory 40 b to the disk drive 200 holding the relevantdata, according to an internal schedule. This disk drive and the writeprocessing are explained using FIG. 6.

(1) The control unit 40 a (CPU) of the control manager 40 creates an FCheader and descriptor in the descriptor area of memory 40 b. Thedescriptor is a command requesting data transfer by a data transfercircuit, and contains the address in cache memory of the FC header, theaddress in cache memory of the data to be transferred, the number ofdata bytes, and the logical address of the disk for data transfer.

(2) The data transfer circuit of the disk adapter 42 is started.

(3) The started data transfer circuit of the disk adapter 42 reads thedescriptor from the memory 40 b.

(4) The started data transfer circuit of the disk adapter 42 reads theFC header from the memory 40 b.

(5) The started data transfer circuit of the disk adapter 42 decodes thedescriptor and obtains the request disk, leading address, and number ofbytes, and reads the data from the cache area of memory 40 b.

(6) After the completion of reading, the data transfer circuit of thedisk adapter 42 transfers the FC header and data to the relevant diskdrive 200 via fiber channel 500 (510). The disk drive 200 writes thetransferred data to an internal disk.

(7) Upon completion of data writing, the disk drive 200 sendsnotification of completion to the data transfer circuit of the diskadapter 42 via the fiber channel 500 (510).

(8) Upon receiving notification of completion, the started data transfercircuit of the disk adapter 42 uses an interrupt to send completionnotification to the control manager 40.

(9) The control unit 40 a of the control manager 40 obtains theinterrupt source of the disk adapter 42 and confirms the writeoperation.

(10) The control unit 40 a of the control manager 40 checks the endpointer of the disk adapter 42 and confirms the completion of the writeoperation.

In both FIG. 5 and FIG. 6, arrows indicate the transfer of data andother packets, and U-shaped arrows represent data reading, indicatingthat data is sent back in response to a data request. Because startingof the control circuit in the DA and confirmation of the end state arenecessary, seven exchanges of signals are necessary between the CM 40and DA 42 in order to perform a single data transfer. Between the DA 42and disk 200, two signal exchanges are required.

Thus it is clear that low latency is required for the connection betweenthe cache control unit 40 and the disk adapter 42, whereas an interfacewith fewer signals can be used between the disk adapter 42 and diskdevice 200.

Next, read/write access of the above-described system disk drives 453,454 is explained. Read/write access from the CM (CPU) 40 a is similar tothat in FIG. 5 and FIG. 6, with DMA transfer performed between thememory 40 b and the system disk drives 453, 454. That is, a DMA circuitis provided in the fiber channel circuit 452 of FIG. 2, and the CPU 400(410) prepares a descriptor and starts the DMA circuit of the fiberchannel circuit 452.

For example, reading of firmware, log data, and backup data (includingdata saved from the cache area) on the system disk drive is similar tothat of FIG. 5; the CPU 400 (410) creates an FC header and descriptor,and by starting the DMA circuit (read operation) of the fiber channelcircuit 452, the firmware, log data, and backup data are transferred byDMA from the system disk drive 453, 454 to the memory 40 b.

Similarly, writing of log data and backup data is similar to that inFIG. 6; the CPU 400 (410) creates an FC header and descriptor, and bystarting the DMA circuit (write operation) of the fiber channel circuit452, log data and backup data are transferred by DMA to the system diskdrive 453, 454 from the memory 40 b. This log data writing is executedperiodically, or when a fixed amount of log data is accumulated in thememory 40 b, or when power is turned off.

By thus incorporating system disks into controllers, even when problemsarise in a path between controllers and the BRTs and disk enclosures, ifthe controller and other paths are normal, firmware and apparatusconfiguration backup data can be read by the controller from the systemdisk, and operations employing other paths are possible. Moreover, acontroller can read and write log data to and from a system disk, sothat analysis upon occurrence of a fault and diagnostics for faultprevention are possible.

Further, when in the event of a power outage the power is switched tobatteries and the data in cache memory is backed up to a system disk,there is no need to supply power to a disk enclosure, so that thebattery capacity can be made small. And, because there is no need towrite backup data to a system disk via a disk adapter or cable, thewrite time can be shortened, so that the battery capacity can be madesmall even for a large write memory capacity.

Further, because a pair of system disk drive is provided in a redundantconfiguration, even if a fault were to occur in one of the system diskdrives, backup using the other system disk drive would be possible. Thatis, a RAID-1 configuration can be adopted.

The service processor 44 of FIG. 2 can also access the system diskdrives 453, 454 via the bridge circuit 450. Firmware and apparatusconfiguration data are downloaded from the service processor 44 to thesystem disk drives 453, 454.

Mounted Configuration

FIG. 7 shows an example of the mounted configuration of control modulesof this invention, and FIG. 8 shows a mounted configuration example,including disk enclosures and the control modules of FIG. 7.

As shown in FIG. 8, on the upper side of the storage apparatus housingare installed four disk enclosures 2-0, 2-1, 2-8, 2-9. Control circuitsare installed in the lower half of the storage apparatus. As shown inFIG. 7, the lower half is divided into front and back by a back panel 7.Slots are provided in the front side and in the back side of the backpanel 7. This is an example of the mounted structure of a storage systemwith eight CMs installed, larger in scale than the four CMs 4-0 to 4-3of FIG. 1; but except for the different number of CMs, the configurationis the same.

That is, as shown in FIG. 7, eight CMs 4-0 to 4-7 are positioned on thefront side, and two FRTs 6-0, 6-1, eight BRTs 5-0 to 5-7, and a serviceprocessor SVC (symbol “44” in FIG. 2) in charge of power supply controland similar, are positioned on the back side.

Two system disk drives 453, 454 are provided in each of the CMs 4-0 to4-7. In FIG. 7, the symbols “453” and “454” are assigned to the systemdisk drives (SDs) of CM 4-0; the configuration is similar for the otherCMs 4-1 to 4-7, but these are omitted in order to avoid complicating thedrawing. These system disk drives 453, 454 can be inserted and removedfrom the back panel 7.

In FIG. 7, the eight CMs 4-0 to 4-7 and two FRTs 6-0, 6-1 are connected,via the back panel 7, to a four-lane PCI-Express bus. The PCI-Expresshas four signal lines (for differential, bidirectional communication) ina lane, so that there are 16 signal lines in four lanes, and the totalnumber of signal lines is 16×16=256. The eight CMs 4-0 to 4-7 and eightBRTs 5-0 to 5-7 are connected via the back panel 7 to fiber channel. Fordifferential, bidirectional communication, the fiber channel has 1×2×2=4signal lines, and there are 8×8×4=256 such signal lines.

Thus by selectively utilizing buses at different connection points, evenin a large-scale storage system, connections between eight CMs 4-0 to4-7, two FRTs 6-0 and 6-1, and eight BRTs 5-0 to 5-7 can be achievedusing 512 signal lines. This number of signal lines can be mountedwithout problem on a back panel board 7, and six signal layers on theboard are sufficient, so that in terms of cost this configuration isfully realizable.

In FIG. 8, four disk enclosures, 2-0, 2-1, 2-8, 2-9 are installed; theother disk enclosures, 2-3 to 2-7 and 2-10 to 2-15, are provided inseparate housings.

Because one-to-one mesh connections are provided between the diskadapters 42 a, 42 b of each of the control modules 4-0 to 4-7 and theBRTs 5-0 to 5-7, even if the number of control modules 4-0 to 4-7comprised by the system (that is, the number of disk adapters 42 a, 42b) is increased, fiber channel with a small number of signal linescomprised by the interface can be employed for connection of the diskadapters 42 a, 42 b to the BRTs 5-0 to 5-7, so that problems arisingfrom mounting can be resolved.

Thus if, for example, system disk drives of size approximately 2.5inches are used, mounting (incorporation) in CM 4-0 and similar iseasily accomplished, and so no problems are posed by mounting.

Log Data Output Method Upon Controller Abnormality

As shown in FIG. 2, by installing the system disk drives 453, 454 in theCM 4-0 and similar, the above-described advantages accrue, but problemsarise which are different from those of an apparatus with system diskdrives installed in disk enclosures, such as in the configuration of theprior art in FIG. 20.

Log data, comprising log data for tasks and threads in progress in eachof the CMs 4-0 to 4-3, is stored in the system disk drives 453, 454 forthe CM. In the conventional configuration of FIG. 20, even if amalfunction occurs in one CM in the system, the other CMs can access thesystem disk drives of the malfunctioning CM, and log data output ispossible.

But as shown in FIG. 2, when the system disk drives 453, 454 areinstalled in CM 4-0, if there is a malfunction due to some problem withthe CM 4-0, there are cases in which the system disk drives 453, 454 ofthe CM 4-0 cannot be accessed; in such cases, log data output is notpossible.

Below, a log data output control method is explained for avoiding statesin which log data output is not possible in the event of a CMabnormality.

FIG. 9 through FIG. 14 explain a method of log data output (log dataoutput method) of one embodiment of the invention. This method is amethod of mounting a system disk drive 454, mounted in the abnormal CM4-0, in a system disk slot of a normally operating CM 4-1, and ofoutputting the log data of the abnormal CM 4-0.

(1) As shown in FIG. 9, when an abnormality occurs in the control module(CM) 4-0, the system disk drive 454-0 installed in the abnormal CM 4-0is removed from the abnormal CM 4-0.

(2) Next, as shown in FIG. 10, a normally operating CM 4-1 is connectedto a maintenance and diagnostics apparatus 8 comprising a personalcomputer, and one of the system disk drives 454-1 installed within theCM 4-1 is separated from the apparatus 4-1 under a data output modeseparation instruction.

(3) Next, as shown in FIG. 11, after completion of separation of thesystem disk drive 454-1 for the CM 4-1, the system disk drive 454-1 isremoved from the apparatus 4-1.

(4) As shown in FIG. 12, the system disk drive 454-0 which had beeninstalled in and then removed from the abnormal CM 4-0 is inserted intothe system disk slot of the normal CM 4-1, from which the system diskdrive has been removed.

(5) As shown in FIG. 13, the CM 4-1 detects the mounting of the systemdisk drive 454-0, and without affecting the log data area CM #0 of theabnormal CM 4-0 in the system disk drive 454-0, the log data CM #1 ofthe normal CM 4-1 in the system disk drive 453 is copied to a log dataspare area for the system disk drive 454-0 by using rebuild/copy backprocessing. By this means, the log data CM #1 of the normal CM 4-1 issubjected to redundancy processing.

(6) As shown in FIG. 14, the maintenance/diagnostics apparatus 8instructs the normal CM 4-1 to acquire the log data CM #0 of theabnormal CM 4-0. The normal CM 4-1 outputs the log data CM #0 in themounted system disk drive 454-0 of the abnormal CM to themaintenance/diagnostics apparatus 8.

By this means, fault analysis of an abnormal CM 4-0 can be performed bythe maintenance/diagnostics apparatus 8, using log data from theabnormal CM.

Because this method does not require equalization processing (copyprocessing) of log data for the system disks of each of the CMs, theprocessing burden can be alleviated. Normally when a disk drive isexchanged, rebuild/copy back processing is executed automatically, sothat the data in the exchanged disk is lost; but in this embodiment,even when the relevant system disk drive is mounted on a different CM, aspare area is specified, so that the log data can be output and faultanalysis can be performed more efficiently.

Log Data Output Processing Upon Controller Abnormality

FIG. 15 shows the flow of log data output processing upon occurrence ofa controller abnormality in one embodiment of the invention, FIG. 16explains the configuration definition table in FIG. 15, FIG. 17 showsthe flow of information extraction processing in FIG. 15, and FIG. 18and FIG. 19 explain the exchange processing of FIG. 15. FIG. 15 showsthe log data output processing for a normal CM 4-1.

(S10) As shown in FIG. 10, the normally operating CM 4-1 receives fromthe connected maintenance/diagnostics apparatus 8 an instruction toseparate one of the system disk drives 454-1 within the CM 4-1, andseparates the system disk drive 454-1 from the apparatus 4-1. As shownin FIG. 11, an attendant or similar then removes the system disk drive454-1 from the apparatus 4-1, and the CM 4-1 detects this removal. Forexample, the output of connector pins is detected.

(S12) As shown in FIG. 12, the system disk drive 454-0, which had beenmounted in and was removed from the abnormal CM 4-0, is inserted intothe system disk slot of the normal CM 4-1 from which the system diskdrive 454-1 has been removed. The normal CM 4-1 monitors the statewithin the FC paths, and detects the insertion of the system disk drive454-0, that is, the connection to a path. The CM 4-1 then reads the WWN(World Wide Name) on the FC map from the system disk drive 454-0.

(S14) Next, the CM 4-1 judges whether the inserted system disk drive hadbeen mounted in an abnormal CM, or is a drive for exchange. To this end,the CM 4-1 references the configuration information definition table 470shown in FIG. 16 (also shown in FIG. 2). As indicated in FIG. 16, theconfiguration information definition table 470 stores the WWNs andabnormality information F for the system disk drives of all CMs and foruser disk drives. For example, as information for the two system diskdrives 453, 454 mounted in CM-0, the World Wide Names WWN-1, WWN-2 andabnormality information F are stored. When for example an abnormalityoccurs in CM 4-0, the CM 4-1 is notified, and the abnormalityinformation fields for the system disk drives of CM 4-0 are set toabnormality. The CM 4-1 employs the previously read WWN described aboveto reference the system disk fields in the configuration informationdefinition table 470, and judges whether the WWN coincides with a WWN ofthe abnormal CM 4-0.

(S16) If the CM 4-1 judges that the previously read WWN coincides with aWWN of the abnormal CM (ID coincidence), it is judged that a system diskdrive 454-0 of a malfunctioning CM has been inserted, the read mode flagis turned on, and processing advances to step S18. If on the other handthe CM 4-1 judges that the WWN which has been read does not coincidewith a WWN of the abnormal CM (ID non-coincidence), it is judged that asystem disk drive for exchange has been inserted, and processingadvances to step S18.

(S18) The CM 4-1 then writes the WWN read for the system disk drive inits own system disk drive field in the configuration informationdefinition table 470.

(S20) The CM 4-1 starts the inserted disk drive, and reads the diskinformation (for example, the vendor name, product name, disk version,and similar).

(S22) The CM 4-1 checks the read mode flag, and if the flag is set toon, performs the information extraction processing of FIG. 17, but ifthe flag is not set to on, performs the exchange processing of FIG. 18and FIG. 19. Processing then ends.

Information extraction processing is explained using FIG. 17. In thisprocessing, the log data area CM #0 of the abnormal CM 4-0 in the systemdisk drive 454-0 explained in the above FIG. 13 is not in any wayaffected, while the log data CM #1 of the normal CM 4-1 in the systemdisk drive 453 is rebuild/copy back processed to a log data spare areaof the system disk drive 454-0, and the log data CM #1 of the normal CM4-1 is made redundant.

(S30) The CM 4-1 begins data equalization processing from its own pairof system disk drives 453. First, the CM 4-1 acquires the disk areainformation for the system disk drive 454-0, and detects the log dataarea CM #0 of the abnormal CM 4-0.

(S32) The CM 4-1 sets the write start position from this log data area(spare log area) of the log data CM #1 for the normal CM 4-1 in thesystem disk drive 453.

(S34) As explained in FIG. 13, the CM 4-1 reads the log data #1 on thesystem disk drive 453, and copies this data to the log data spare areaof the system disk drive 454-0, to render redundant the log data CM #1of the normal CM 4-1.

Next, the maintenance/exchange processing of FIG. 15 is explained usingFIG. 18 and FIG. 19. Maintenance/exchange processing is processing torender redundant a normal system disk drive, and is primarily performedfor maintenance purposes. That is, copy back operation is performed toan exchange disk with the same layout as the normal CM system disk.

As shown in FIG. 18, the system disk drive 454 in question is removedfrom CM 4-1, and an exchange disk drive 454-N is inserted into thesystem disk slot from which the system disk drive 454 was removed. Then,after detection of mounting of the exchange system disk drive 454-N asindicated in FIG. 19, copy back processing from the normal system diskdrive 453 is used to render the log data redundant. When log dataredundancy processing is completed, normal operation is initiated.

In this way, even if system disk drives are built into controllers, thelog data on system disks in an abnormal controller can be output.Further, log data redundancy processing is performed by normalcontrollers, so that a normal controller can perform log data redundancyprocessing using a pair of system disk drives.

Because there is no need to perform equalization processing of log dataon the system disks of each controller, the burden of log dataequalization processing can be alleviated. Moreover, even when a systemdisk drive is mounted on another controller, loss of the log data forthe anomalous controller can be prevented.

Other Embodiments

In the above-described embodiment, log data output processing wasexplained for an example of two control modules; but similar applicationis possible when there are three or more control modules. The number ofchannel adapters and disk adapters within control modules can beincreased or decreased as necessary.

As the disk drives, hard disk drives, optical disc drives,magneto-optical disc drives, and other storage devices can be employed.Further, the configuration of the storage system and controllers(control modules) is not limited to that of FIG. 1, and application toother configurations (such as for example that of FIG. 20) is possible.

In the above, embodiments of this invention have been explained, butvarious modifications can be made within the scope of the invention, andthese modifications are not excluded from the scope of the invention.

Because system disks are incorporated into control modules, even ifproblems occur in a path between a control module and a disk storagedevice, a control module and another path can be used to read firmwareand apparatus configuration backup data from a system disk, andoperation using other paths is possible; further, log data can be readand written, so that analysis upon occurrence of a fault and diagnosticsfor fault prevention are possible.

Moreover, even when an abnormality occurs in one control module, asystem disk drive of the one control module can be inserted into anothercontrol module, and data for the one control module can be read, so thateven when system disk drives are incorporated into control modules, thelog data on a system disk of an anomalous control module can be output.Consequently a storage system with high reliability can be provided.

1. A data storage system, comprising: a plurality of disk storagedevices which store data; and a plurality of control modules, connectedto said plurality of disk storage devices, which control access to saiddisk storage devices according to access instructions from ahigher-level system, wherein each of said control modules comprises: amemory, having a cache area which stores a portion of the data stored bysaid disk storage devices; a control unit, which performs said accesscontrol; and a pair of system disk units, connected to said controlunit, which store, at least the log data of said control unit; andwherein one control module, upon occurrence of an abnormality in anotherof said control modules, detects that one system disk unit of saidanother control module has been inserted into a system disk slot of saidone control module, incorporates said one system disk unit of saidanother control module, and outputs log data of said one system diskunit of said another control module.
 2. The data storage systemaccording to claim 1, wherein said one control module, afterincorporating said one system disk unit of said another control module,copies log data of the system disk unit of said one control module tosaid incorporated system disk unit, without destroying log data of saidanother control module on the system disk unit of said another controlmodule.
 3. The data storage system according to claim 1, wherein saidone control unit reads an identifier of said inserted system disk unit,and judges whether said one system disk unit of said another controlmodule has been incorporated.
 4. The data storage system according toclaim 3, wherein said one control unit, upon judging from saididentifier that said inserted system disk unit is not one system diskunit of said another control module, copies log data of a system diskunit of said one control module to said incorporated system disk unit.5. The data storage system according to claim 2, wherein said onecontrol module reads the log data area of said incorporated system diskunit of said another control module, and copies, to an area other thansaid log data area of the system disk unit of said other control module,log data of the system disk unit of said one control module.
 6. The datastorage system according to claim 1, wherein said one control moduleseparates another system disk unit of said one control module inresponse to an instruction of an external apparatus, and releases asystem disk slot to enable insertion of a system disk unit of saidanother control module.
 7. The data storage system according to claim 1,wherein said one control module outputs log data of said another controlmodule in said incorporated system disk unit, in response to a log dataacquisition instruction from an external apparatus.
 8. The data storagesystem according to claim 7, wherein said one control module outputs thelog data of said other control module to said external apparatus.
 9. Alog data output method upon an abnormality for a storage controlapparatus having a plurality of control modules, connected to aplurality of disk storage devices which store data, which control accessto said disk storage devices according to access instructions from ahigher-level system, and in which each of said control modules has amemory having a cache area which stores a portion of the data stored bysaid disk storage devices, a control unit which performs said accesscontrol, and a pair of system disk units, connected to said controlunit, which store at least the log data of said control unit, comprisingthe steps of: detecting, at the time of an abnormality in anothercontrol module, an insertion into a system disk slot of one controlmodule of one system disk unit which has been removed from said anothercontrol module; incorporating said inserted one system disk unit of saidanother control module into said one control module; and outputting,using said one control module, log data of said incorporated one systemdisk unit of said another control module.
 10. The log data output methodupon an abnormality for a storage control apparatus according to claim9, further comprising a step, after said one system disk unit of saidanother control module has been incorporated, of copying log data of asystem disk unit of said one control module to said incorporated systemdisk unit, without destroying log data of said another control module inthe system disk unit of said another control module.
 11. The log dataoutput method upon an abnormality for a storage control apparatusaccording to claim 9, wherein said detection step comprises: a step ofreading an identifier of said inserted system disk unit; and a step ofjudging whether said one system disk unit of said another control modulehas been incorporated.
 12. The log data output method upon anabnormality for a storage control apparatus according to claim 11,further comprising a step, when said inserted system disk unit is judgedfrom said identifier not to be said one system disk unit of said anothercontrol module, of copying log data of a system disk unit of said onecontrol module to said incorporated system disk unit.
 13. The log dataoutput method upon an abnormality for a storage control apparatusaccording to claim 10, wherein said copy step comprises: a step ofreading the log data area of said incorporated system disk unit of saidanother control module; and a step of copying the log data of the systemdisk unit of said one control module to an area other than said log dataarea of said system disk unit of said another control module.
 14. Thelog data output method upon an abnormality for a storage controlapparatus according to claim 9, further comprising: a step of separatinganother system disk unit of said one control module, according to aninstruction from an external apparatus; and a step of releasing a systemdisk slot to enable insertion of a system disk unit of said anothercontrol module.
 15. The log data output method upon an abnormality for astorage control apparatus according to claim 9, wherein said output stepcomprises a step of outputting log data of said another control moduleon said incorporated system disk unit, according to a log dataacquisition instruction from an external apparatus.
 16. The log dataoutput method upon an abnormality for a storage control apparatusaccording to claim 15, wherein said output step comprises a step ofoutputting the log data of said another control module to said externalapparatus.